Two-dimensional variable length coding (2D-VLC) includes collecting or assuming the statistics of two dimensional block transform coefficient events that are each a run of the most likely-to-occur amplitude, e.g., 0, followed by another amplitude. The coding includes assigning variable length codes, e.g., optimal codes such as Huffman codes or Arithmetic codes, to each event. In the description herein, 0 is assumed to be the most likely-to-occur amplitude. The collecting of or assuming statistics includes tracking the quantized non-zero-valued coefficient amplitudes and the number of zero-valued coefficients preceding the non-zero amplitude, i.e., tracking the runlengths of zeros which precedes any non-zero amplitude along a specified path, e.g., a zigzag scan path for a block of coefficients, e.g., an 8 by 8 or a 16 by 16 coefficient block. Table 1 below shows by example the statistics tabulated as a two dimensional table:
TABLE 12D-VLC statisticsRunlength of preceding 0's0123456. . .. . .Coeff.1S10S11S12S13S14S15S16. . .. . .Amp.2S20S21S22S23S24S25S26. . .. . .3S30S31S32S33S34S35S36. . .. . .4S40S41S42S43S44S45S46. . .. . .5S50S51S52S53S54S55S56. . .. . .6S60S61S62S63S64S65S66. . .. . .7S70S71S72S73S74S75S76. . .. . .8S80S81S82S83S84S85S86. . .. . .9S90S91S92S93S94S95S96. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .
In the table, Sij is the likelihood expressed, for example, as a relative number of occurrences of an amplitude of i, i=1, 2, . . . occurring after a run of j 0's, j=0, 1, 2, . . .
A variable length code such as an optimal code is then assigned to each of the events that have an Sij above, with the most likely-to-occur element—typically S10 for the case of encoding a block of transform coefficients in transform coding—having the shortest number of bits, and the least occurring event coded using the longest number of bits. Table 2 below shows an example of a 2D-VLC table:
TABLE 22D-VLC codesRunlength of preceding 0's0123456. . .. . .Coeff.1C10C11C12C13C14C15C16. . .. . .Amp.2C20C21C22C23C24C25C26. . .. . .3C30C31C32C33C34C35C36. . .. . .4C40C41C42C43C44C45C46. . .. . .5C50C51C52C53C54C55C56. . .. . .6C60C61C62C63C64C65C66. . .. . .7C70C71C72C73C74C75C76. . .. . .8C80C81C82C83C84C85C86. . .. . .9C90C91C92C93C94C95C96. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .. . .
where Cij is the codeword used to encode the event of the combination of j consecutive 0-valued coefficients followed by a single non-zero coefficient of amplitude i, j=0,1, . . . and i=1, 2, . . .
2D-VLC is used in common transform coding methods such as JPEG, MPEG1, MPEG2, ITU-T-261, etc., as follows. For motion video, an image is divided into blocks, e.g., 8 by 8 or 16 by 16 blocks. Each image is classified as interframe or intraframe. Interframe images are typically post motion compensation. The blocks of the image are transformed and the transform coefficients are quantized. The quantized transform coefficients are then coded along a specified path according to a 2D-VLC table. Typically, interframe and intraframe images have different 2D-VLC tables. The DC component is typically separately encoded. Furthermore, the 2D-VLC table may be truncated so that the least frequently occurring events use an escape code followed by a fixed length code. A special “EOB” code is used to indicate the end of a block when all remaining coefficients are zero.
Still images are similarly encoded, e.g., in the same manner as an intraframe image for motion video.
A table lookup may be used to implement a 2D-VLC scheme. Prior to the table look up, the runlength of zero amplitudes preceding any non-zero amplitude and the non-zero amplitude are determined. The table look up uses a 2D table for those likely events encoded using variable length encoding. An escape code together with a fixed length code is used for relatively less likely-to-occur combinations.
The advantage of 2D-VLC is that both the position of each non-zero-valued coefficient as indicated by the runlength, and the quantized amplitude value are coded simultaneously as a pair using one 2D-VLC table. This may result in shorter codes, i.e., codes that use fewer bits than using separate VLC tables for each non-zero-valued coefficient and for its amplitude.
Because of the widespread use of image coding, many patents have been issued on different forms of VLC. U.S. Pat. No. 4,698,672 issued Oct. 6, 1987 to Wen-hsiung Chen, one of the inventors of the present invention, for example described one form of a two-dimensional variable length coding method.
Extensions and variations to the common 2D-VLC method are known. For example, the ITU H.263 compression standard defines one such variation sometimes called three-dimensional VLC (3D-VLC). See PCT patent publication WO 9318616 published Sep. 16, 1993 titled PICTURE DATA ENCODING METHOD and also the ITU-T H.263 standard. In 3D-VLC, each symbol (“event”) is a triplet (LAST, RUN, LEVEL) that includes: LAST, a binary flag that indicates whether or not the current non-zero amplitude-value is the last non-zero coefficient in the block, RUN, the run-length of zero-value coefficients that precede the current non-zero amplitude, i.e., the number of zeroes since the last non-zero coefficient amplitude, and LEVEL, the current non-zero coefficient amplitude value. Thus, there is no need for a separate EOB codeword; whether or not the non-zero coefficient is the last one is incorporated into the event. A table lookup may be used to implement 3D-VLC.
One deficiency of 2D-VLC is that every non-zero-valued coefficient needs to be accompanied by a runlength code to identify its position, in the form of the number of preceding zero-valued coefficients.
In block based transform coding, the inventors have observed that there often is a region, e.g., a low frequency region along the ordering in which non-zero-valued coefficients tend to cluster, i.e., there are often a number of consecutive non-zero-valued coefficients along the low frequency region of the pre-determined path. This may especially occur in intraframe coding and high bit rate interframe coding. Each one of a number of such consecutive non-zero-valued coefficients would require the same number of codewords representing the position and amplitude. That is, 2D-VLC requires a separate runlength code, e.g., C10, C20, C30 . . . , etc., for each of the consecutive non-zero coefficients.
U.S. patent application Ser. No. 10/342,537 to inventors Chen et al., filed Jan. 15, 2003 and titled AN EXTENSION OF TWO-DIMENSIONAL VARIABLE LENGTH CODING FOR IMAGE COMPRESSION describes a method called the “Extended 2D-VLC Method” herein that includes encoding repetitions of some non-zero coefficient values. One variant of the Extended 2D-VLC method provides codes for all the possible amplitude variations of consecutive coefficients that follow a set of zero-valued coefficients. This effectively reduced the runlength to 1 for all cases. The difficulty of this approach is that there are enormous numbers of patterns that can be generated from the amplitudes of consecutive coefficients. For example, with 32 quantization levels as defined in many common video coding standards, there are in the order of 32n patterns that can be generated from n consecutive coefficients. As such, in a practical implementation, only a limited number of the most likely-to-occur non-zero amplitude values, such as 1 and 2, and a limited number of lengths of consecutive non-zero-values, such as 3 or 4 consecutive values, are regrouped for pattern matching.
Furthermore, in coding, while there may be a region where there are clusters of non-zero-valued coefficients, there is also likely to be a high frequency region where any non-zero-valued coefficients are likely to be scattered.
With these observation in mind, the Basic Hybrid VLC Method of above-mentioned incorporated by reference U.S. patent application Ser. No. 10/869,229, now U.S. Pat. No. 7,454,076 to inventors Chen et al. was developed to encode the position and amplitude of quantized transform coefficients separately and takes advantage of the nature of the distribution of the transform coefficients in the low frequency and high frequency regions.
The Extended Hybrid VLC Method of incorporated by reference U.S. patent application Ser. No. 10/898,654, now U.S. Pat. No. 7,483,584 provides an alternative coding method for the high frequency region by taking advantage of the very few amplitude values in the high frequency region, especially, for example, for low bit rate and interframe applications.
In one embodiment of the above-mentioned Basic Hybrid VLC Method, two independent types of coding schemes are introduced to code the quantized coefficients along the path. A boundary is established along the path to define two regions, e.g., a low frequency region and a high frequency region. The boundary can be made adaptive to the video depending on a number of factors such as intraframe coding or interframe coding, standard definition television (SDTV) or high definition television (HDTV), complex scene or simple scene, high bit rate coding or low bit rate coding, and so forth. In one embodiment, the encoding of the quantized coefficients in the low-frequency region includes coding the positions of consecutive non-zero-valued coefficients and the positions of consecutive zero-valued coefficients using a run-length coding method of a first type and a run-length coding method of a second type. The encoding further includes coding the amplitude values and sign of the non-zero-valued coefficients. In the high-frequency region, in one embodiment, the encoding of coefficients in the high frequency region includes encoding the positions of either no consecutive zero-valued coefficients or runs of one or more consecutive zero-valued coefficients using a run-length coding method of a third type. The encoding further includes coding the amplitude values and sign of the non-zero-valued coefficients.
In one embodiment of the above-mentioned Extended Hybrid VLC Method, a coding method is used in the second region that takes into account that almost all non-zero-valued coefficients in the high frequency region are ±1. No amplitude coding is needed to encode runs of consecutive zeroes that end in a coefficient of amplitude 1. An exception (escape) code is included to encode those rare non-zero-valued coefficients that have values other than ±1.
In the Basic Hybrid VLC Method and the Extended Hybrid VLC Method, the consecutive non-zero-valued coefficients and the consecutive zero-valued coefficients in the low frequency region are coded alternatively using two independent one-dimensional variable length coding methods, e.g., using two independent one-dimensional VLC tables. An observation was made that an improvement in coding efficiency can further be achieved by pairing the consecutive non-zero-valued coefficients and zero-valued coefficients as a pair and applying a single two-dimensional table to code the pair. With this observation, the 2-D Non-Zero/Zero Cluster Coding Method of above-mentioned incorporated by reference U.S. patent application Ser. No. 10/922,508, now U.S. Pat. No. 7,471,840 was introduced to improve the coding efficiency, for example for the low frequency region, and in other embodiments for more than the low frequency region.
In one embodiment of the 2-D Non-Zero/Zero Cluster Coding Method, a method includes, in a first contiguous region, identifying events that each include a run of zero-valued coefficients preceding a run of one or more non-zero-valued coefficients. The method includes for each such event, jointly encoding the runlengths of the preceding run of zero-valued coefficients and the following run of non-zero-valued coefficients with a codeword, such that for at least some events, relatively more likely-to-occur pairs of runlengths are encoded by a shorter codeword than relatively less likely-to-occur runlengths. The method further includes encoding each amplitude in the run of consecutive non-zero-valued coefficients, and encoding the signs of such coefficients. In an improved variation, each event includes a single zero-valued coefficient following the run of non-zero-valued coefficients.
In each of the 2-D Non-Zero/Zero Cluster Coding Method, the Basic Hybrid VLC Method, and the Extended Hybrid VLC Method, various variable length coding methods are introduced to encode the relative positions of the clustered or non-clustered transform coefficients. After each such encoding, a coding of the magnitude of each non-zero valued coefficient is included, as is a sign bit (+ or −).
The inventors have noticed that encoding the amplitudes takes up a significant part of the code in VLC coding of clusters of non-zero-valued coefficients. With this in mind, the inventors observed that, at least in theory, an improvement in amplitude code can be achieved by introducing a single multi-dimensional code, say an n-dimensional code, n an integer greater than 1, to encode n clustered non-zero coefficients, instead of using n separate one dimensional codes. The Basic Multi-Dimensional Amplitude Coding Method of above-mentioned incorporated-by-reference U.S. patent application Ser. No. 10/922,507, now U.S. Pat. No. 7,492,956 includes such multidimensional amplitude coding.
One embodiment of the Basic Multi-Dimensional Amplitude Coding Method includes, in a first region, identifying events that each includes a run of one or more non-zero-valued coefficients, and for each such event, encoding the event with a codeword such that for at least some events, relatively more likely-to-occur events are encoded by a shorter codeword than relatively less likely-to-occur events, and for each identified event, jointly encoding a plurality of consecutive values in the run of consecutive non-zero-valued coefficients, the joint encoding according to an amplitude coding method. The method is such that relatively short codewords are formed to represent values or sequences of values that are relatively more likely-to-occur, and relatively long codewords are formed to represent values or sequences of values that are relatively less likely-to-occur. The method is applicable to encoding a region in the series where there is likely to be a cluster of non-zero-valued coefficients.
While the Basic Multi-Dimensional Amplitude Coding Method invention described in U.S. patent application Ser. No. 10/922,507, now U.S. Pat. No. 7,492,956 appears to improve the overall coding efficiency, it was observed that the size of the n-dimensional table used for the joint encoding can become rather large for a large “n.” As a result, in practice, the size of n has to be limited to a low number of consecutive non-zero-amplitude values, such as 1, 2 and 3 for practical implementation.
With this in mind, the Multi-Table Amplitude Coding Method of above-mentioned incorporated by reference U.S. patent application Ser. No. 11/069,622, now U.S. Pat. No. 7,499,596 was introduced. Rather than using a single multidimensional coding table for a cluster of a number—say n—consecutive non-zero-valued coefficients, events are identified within the cluster that each include a run of consecutive amplitude-1 coefficients, followed by a single coefficient of amplitude greater than 1. Included are events of only a single coefficient of amplitude greater than 1 and runs of only amplitude 1. For each event, a codeword is assigned to the runlength of the preceding run of amplitude-1 coefficients combined with the amplitude of the ending coefficient. A two-dimensional coding table is used for each cluster length n, so that the multidimensional table of the Basic Multi-Dimensional Amplitude Coding Method is replaced by a number of increasingly large 2-D coding tables. The value of n can be as large as the position of the breakpoint. One view of the Multi-Table Amplitude Coding Method is that it applies a modified 2D-VLC method within each cluster of consecutive non-zero-valued coefficients, with the most likely to occur amplitude in the cluster being 1, so that, within each cluster, one can view the method as applying a 2D-VLC method to a modified sequence of coefficients, with each coefficient amplitude reduced by 1, and with appropriate assumed or measured statistics for such clusters.
To further improve the coding efficiency, the position code and amplitude code can be jointly coded. One aspect of the Joint Position and Amplitude VLC Method” of above-mentioned U.S. patent application Ser. No. 11/069,621, now U.S. Pat. No. 7,499,595 is jointly encoding the relative position and runlength of each clusters of non-zero valued coefficients with the amplitudes of the non-zero-coefficients in the cluster to form a joint codeword for the combination of the relative position of the cluster and of the non-zero amplitudes within the cluster. In particular, one aspect of the present invention is that rather than concatenating the codes for the position of clusters with the codes for the amplitudes of the non-zero coefficients in the clusters, a function of the position on the one hand, and non-zero amplitudes on the other hand, is used to obtain a single codeword for the position and non-zero-coefficient amplitudes of the cluster. In one embodiment, the signs of the non-zero amplitudes are included such that the function is also of the signs of the non-zero amplitudes.
To further improve the coding efficiency, the position code and amplitude code can be jointly coded. One aspect of the Joint position and Amplitude VLC Method” of above-mentioned U.S. patent application Ser. No. 11/069,621 is jointly encoding the relative position and runlength of each clusters of non-zero valued coefficients with the amplitudes of the non-zero-coefficients in the cluster to form a joint codeword for the combination of the relative position of the cluster and off the non-zero amplitudes within the cluster. In particular, one aspect of the present invention is that rather than concatenating the codes for the position of clusters with the codes for the amplitudes of the non-zero coefficients in the clusters, a function of the position on the one hand, and non-zero amplitudes on the other hand, is used to obtain a single codeword for the position and non-zero-coefficient amplitudes of the cluster. In one embodiment, the signs of the non-zero amplitudes are included such that the function is also of the signs of the non-zero amplitudes.
As an example, the two-dimensional position code and the multi-dimensional amplitude code, e.g., n-dimensional amplitude code, were n is an integer greater than 1 may be jointly coded as a (2+n)-dimensional code in the low frequency region, while a one-dimensional position code and one-dimensional amplitude code are jointly coded as 2-dimensional code, such as the conventional 2D-VLC, in the high frequency region. While it is relatively easy to construct a 2D-VLC code table for the high frequency coefficients, a construction of (2+n) dimensional code table for the low frequency coefficients simply becomes too large to be manageable.
In order to reduce the size of the code table to a manageable size in the joint position and amplitude coding, the size of n needs to be restricted to a low number. With this restriction, those clustered coefficients with a large n can always be resorted back to a separate position and amplitude coding.
There is always a need to improve the coding efficiency of variable length coding of quantized transform coefficients that occur in transform image compression. While each of the techniques described above provides a potential improvement, the inventors have found that there is still room for improvement.